1. Field of the Invention
The present invention generally relates to a vehicle-applied rear-and-side monitoring system and vehicle-applied rear-and-side monitoring and alarming system and more particularly, to a vehicle-applied rear-and-side monitoring system, which monitors a rear-and-side of a running vehicle, i.e. own vehicle, such as a motor vehicle by taking an image of a rear-and-side road view of the own vehicle by an image taking means such as a video camera and the like set on the own vehicle and by detecting a following vehicle being approaching the own vehicle from the rear-and-side with use of the taken road image, and to a vehicle-applied rear-and-side monitoring and alarming system, which alarms a driver when the above monitoring system have detected a danger.
2. Description of the Related Art
For example, Japanese Patent No. 2641562 (hereinafter JP '562), Japanese Patent Application Laid-open No. 6-107096 (hereinafter JP '096) and also Laid-open No. 7-50769 (hereinafter JP '769) disclose a vehicle-applied rear-and-side monitoring and alarming system, which correctly recognizes a neighboring traffic lane of a traffic lane on which an own vehicle is running, and which gives a driver of the own vehicle an alarm of dangerousness of a collision when the driver is going to change the lane in case that a following vehicle is running on the neighboring lane, thereby avoiding a collision with the following vehicle.
The above system recognizes a lane marking on the road so as to distinguish traffic lanes with use of a road image taken by a video camera and sets a monitoring area on the neighboring lane. This system is intended to reduce an image-processing amount for detecting the following vehicle in the neighboring lane, i.e. in the monitoring area, so that the following vehicle in the neighboring lane can be detected fast and surely.
JP '096 and JP '769 disclose the system applying an optical flow, which will be briefly described with reference to FIGS. 13 and 14.
FIG. 13 is a block diagram showing a prior art vehicle-applied rear-and-side monitoring and alarming system, wherein 10 indicates an image taking portion used as an image taking means and has a video camera 11 for example. And, 30 is a data processing portion use as an arithmetical unit, and 42 is a speaker use as an alarming means.
The image taking portion 10 is arranged at a required position on a rear side, e.g. over the trunk room, of the own vehicle for taking a rear-and-side road view of the vehicle. The data processing portion 30 has CPU 31 (i.e. a central processing unit) to operate according to a program, ROM 32 to keep the program for the CPU 31 and a preset value, and RAM 33 to temporary keep required data for calculation by the CPU 31. The speaker 42 is arranged inside the own vehicle and raises a voice or an alarm sound on the basis of a signal from the CPU 31 of the data processing portion 30 so as to have the driver take notice of the danger.
FIGS. 14A-14C are explanatory illustrations showing change of a rear-and-side road image taken by the image taking portion 10. FIG. 14A is of at a time t, and FIG. 14B is of at a time t+.DELTA.t.
In these figures, 200 indicates a following vehicle running at the rear-and-side of the own vehicle, 300 indicates a road sign standing on one side of the road 500, and 400 indicates a building neighboring the road 500.
When the own vehicle is running straight on a plain road, the road sign 300 and the building 400 go away and therefore their images become small, as time goes by. That is, the road sign 300 and the building 400 in the image of FIG. 14B are smaller than those shown in FIG. 14A.
Description is given below to an optical flow with reference to these figures.
There can be a plurality of subject points, for example, on the following vehicle 200, the road sign 300, and the building 400 such as 201a,202a,301a,302a,303a,401a and 402a at the time t in FIG. 14A and 201b,202b,301b,302b,303b,401b and 402b at the time t+.DELTA.t in FIG. 14B. Speed vectors connecting the corresponding subject points are obtained as optical flows such as 201F,202F,301F,302F,303F,401F and 402F shown in FIG. 14C.
Here, the optical flow is acknowledged to be radially formed from an infinity point in the road image, which infinity point is defined as FOE (i.e. Focus of Expansion). In an image wherein the own vehicle is running straight, the FOE is positioned at the opposite point to the running direction of the vehicle.
Optical flows of objects being going away from the own vehicle become vectors to converge on the FOE, and optical flows of objects being approaching to the own vehicle become vectors to diverge from the FOE.
Accordingly, since the optical flows 201F and 202F of the following vehicle 200 are vectors in a diverging direction, the following vehicle is judged to be approaching to the own vehicle.
Length of the optical flow is in proportion to a distance per unit time, i.e. speed difference, between the own vehicle and the object and also in proportion to a distance itself between the own vehicle and the object. This will be described with reference to FIG. 15.
FIG. 15 is a theoretical scheme showing an optical arrangement, wherein 11a indicates a lens of the video camera of the image taking portion 10, 11b indicates an image plane of the video camera, f indicates a distance from the lens 11a to the image plane 11b, P(Xa,Ya,Za) indicates a point on the following vehicle, and p(xa,ya) on the image plane 11b indicates a corresponding point to the point P.
Based on the above scheme in FIG. 15, a following equation is obtained due to the similar triangles. EQU xa=f.multidot.Xa/Za (1)
By differentiating the equation (1) an equation (2) is obtained. EQU Xa'=(.DELTA.xa/.DELTA.t.multidot.Za+xa.multidot.Za')/f (2)
The x component u of the optical flow is obtained as follows: EQU u=.DELTA.xa/.DELTA.t (3)
Then, from the equation (3): EQU Za=(f.multidot.Xa'-xa.multidot.Za')/u (4)
Za' in the equation (4) is a speed difference or a relative speed between the following vehicle 200 (FIG. 14A) and the own vehicle.
By replacing the relative speed Za' with "-.alpha." the equation (4) is becomes: EQU Za=(f.multidot.Xa'+xa.multidot..alpha.)/u (5)
Accordingly, the x component u of the optical flow can be given with the following equation (6). EQU u=(f.multidot.Xa'+xa.multidot..alpha.)/Za (6)
Ya, i.e. the Y-axis of the point P, can be obtained similarly.
Accordingly, based on the equation (6), the smaller the above Z is, that is, the smaller the distance to the following vehicle 200 is, or the larger the above .alpha. is, that is, the larger the relative speed is, the larger the x component of the optical flow is. This also applies to Y-direction.
Then, the smaller the distance to the following vehicle 200 is and further the larger the relative speed is, the longer the optical flow is, and then the optical flow diverges from the FOE. It can be understood that the longer the optical flow is, the bigger the danger with the following or adjoining vehicle is.
Accordingly, the data processing portion 30 judges that the object on the road exists near the own vehicle or is approaching thereto with a high speed when the optical flow is a vector in a diverging direction and simultaneously is large, and that degree of danger is high.
And, when degree of danger is judged high, the speaker 42 makes a driver know the situation.
Optical flows of objects on the image can be obtained similarly to the above, and thereby degree of danger with each of the objects can be judged. And, a driver can be given an alarm according to the degree of danger, thereby preventing a dangerous state or an actual trouble.
In the prior art, as shown in FIG. 16, the monitoring area is separated, by lane markings defining the own vehicle running lane, to an area of the own vehicle running lane and an area of the neighboring lanes on both sides of the own vehicle running lane so that required processing time can be decreased thereby to attain high speed processing. And, the FOE is detected by extending the lane markings, and the following vehicle 200 in the own vehicle running lane area or in the neighboring lane area is detected by obtaining its optical flow, and degree of danger with the following vehicle 200 running at the rear or on the neighboring lanes is judged according to length of the optical flow.
Among currently proposed methods of detecting the optical flow, one having reached to the practical level is the correlation method, which requires enormous calculation since a corresponding point of each of the picture elements in the image is searched for calculating a correlation value. The method has, however, an advantage that the optical flow can be relatively correctly obtained even though the image is complicated.
More specifically, in a general correlation method, when the optical flow is obtained from the image at a time T, the corresponding point of each of all the picture elements in the image at the time T have to be searched in the image at the time T-.DELTA.T, which requires enormous calculation and simultaneously has a problem of bringing about a mis-correspondence.
Therefore, a monitoring area is considered to be set in order to decrease the processing time and to raise an accuracy of the detection.
With respect to the vehicle-applied rear-and-side monitoring and alarming system disclosed in JP '562, since the monitoring area is set with use of the lane markings detected and recognized in the road image taken by the video camera, processing of recognition of the lane markings needs relatively long time, thereby limiting possibility of shortening the processing time.
Especially, there is much difficulty in recognizing lane markings of the curved road.
Further, the monitoring area could not be set during night or a rainy day since lane markings is difficult to be recognized.